Off-policy and on-policy reinforcement learning with the Tsetlin machine

Abstract The Tsetlin Machine is a recent supervised learning algorithm that has obtained competitive accuracy- and resource usage results across several benchmarks. It been used for convolution, classification, regression, producing interpretable rules in propositional logic. In this paper, we introduce the first framework reinforcement based on Machine. Our integrates value iteration with regression as function approximator. To obtain accurate off-policy state-value estimation, propose modified feedback mechanism adapts to dynamic nature of iteration. particular, show able unlearn recover from misleading experiences often occur at beginning training. A key challenge address mapping intrinsically continuous architecture, leveraging probabilistic updates. While off-policy, learns significantly slower than neural networks on-policy. However, by introducing multi-step temporal-difference combination high-frequency logic patterns, are close performance gap. Several gridworld instances document our can outperform comparable network models, despite being simple one-level AND-rules Finally, how class models learnt problem be translated into more understandable graph structure. structure captures approximation corresponding policy found